Beyond 2D%2Dgrids a dependence maximization view on image browsing.

A Bayesian Review of the Poisson%2DDirichlet Process
The two parameter Poisson-Dirichlet process is also known as the Pitman-Yor Process and related to the ChineseRestaurant Process, is a generalisation of the Dirichlet Process, and is increasingly being used for probabilistic modelling in discrete areas such as language and images. This article reviews the theory of the Poisson-Dirichlet process in terms of its consistency for estimation, the convergence rates and the posteriors of data. This theory has been well developed for continuous distributions (more generally referred to as nonatomic distributions). This article then presents a Bayesian interpretation of the Poisson-Dirichlet process: it is a mixture using an improper and infinite dimensional Dirichlet distribution. This interpretation requires technicalities of priors, posteriors and Hilbert spaces, but conceptually, this means we can understand the process as just another Dirichlet and thus all its sampling properties fit naturally. Finally, this article also presents results for the discrete case which is the case seeing widespread use now in computer science, but which has received less attention in the literature.
Unsupervised Object Discovery A Comparison.

A segmented topic model based on the two%2Dparameter Poisson%2DDirichlet process.

Machine Learning and Knowledge Discovery in Databases
Knowledge discovery in databases is not only the nontrivial extraction of implicit, previously unknown and potentially useful information from databases. We argue that in contrast to machine learning, knowledge discovery in databases should be applied to real world databases. Since real world databases are known to be very large, they raise problems of the access. Therefore, real world databases only can be accessed by database management systems and the number of accesses has to be reduced to a minimum. Considering this property, we are forced to use, for example, standard set oriented interfaces of relational database management systems in order to apply methods of knowledge discovery in databases. We present a system for discovering data dependencies, which is build upon a set oriented interface. The point of main effort has been put on the discovery of value restrictions, unary inclusion- and functional dependencies in relational databases. The system also embodies an inference rel...
Machine Learning and Knowledge Discovery in Databases
Knowledge discovery in databases is not only the nontrivial extraction of implicit, previously unknown and potentially useful information from databases. We argue that in contrast to machine learning, knowledge discovery in databases should be applied to real world databases. Since real world databases are known to be very large, they raise problems of the access. Therefore, real world databases only can be accessed by database management systems and the number of accesses has to be reduced to a minimum. Considering this property, we are forced to use, for example, standard set oriented interfaces of relational database management systems in order to apply methods of knowledge discovery in databases. We present a system for discovering data dependencies, which is build upon a set oriented interface. The point of main effort has been put on the discovery of value restrictions, unary inclusion- and functional dependencies in relational databases. The system also embodies an inference rel...
Estimating Likelihoods for Topic Models.

Kernel Conditional Quantile Estimation via Reduction Revisited.
Abstract—Quantile regression refers to the process of estimating the quantiles of a conditional distribution and has many important applications within econometrics and data mining, among other domains. In this paper, we show how to estimate these conditional quantile functions within a Bayes risk minimization framework using a Gaussian process prior. The resulting non-parametric probabilistic model is easy to implement and allows non-crossing quantile functions to be enforced. Moreover, it can directly be used in combination with tools and extensions of standard Gaussian Processes such as principled hyperparameter estimation, sparsification, and quantile regression with input-dependent noise rates. No existing approach enjoys all of these desirable properties. Experiments on benchmark datasets show that our method is competitive with state-of-the-art approaches. Keywords-Regression; Quantile Regression; Gaussian Processes; I.
Exploring Scale%2DInduced Feature Hierarchies in Natural Images.

Guest editors%27 introduction special issue of selected papers from ECML PKDD 2009.

Guest editors%27 introduction Special Issue from ECML PKDD 2009.

Product retrieval for grocery stores.

Natural language retrieval of grocery products.

Product retrieval for grocery stores.

ALVIS %2D Superpeer Semantic Search Engine %2D ECDL 2006 Demo Submission.

Open source search and research.

SIGIR06 workshop report Open Source Information Retrieval systems %28OSIR06%29.
This paper summarizes the activities and discussions of the 2006
A temporally adaptive content%2Dbased relevance ranking algorithm.

Discrete Component Analysis.
This article presents a unified theory for analysis of components  in discrete data, and compares the methods with techniques such as  independent component analysis, non-negative matrix factorisation and  latent Dirichlet allocation. The main families of algorithms discussed  are a variational approximation, Gibbs sampling, and Rao-Blackwellised  Gibbs sampling. Applications are presented for voting records from the  United States Senate for 2003, and for the Reuters-21578 newswire collection.
Discrete Component Analysis.
This article presents a unified theory for analysis of components  in discrete data, and compares the methods with techniques such as  independent component analysis, non-negative matrix factorisation and  latent Dirichlet allocation. The main families of algorithms discussed  are a variational approximation, Gibbs sampling, and Rao-Blackwellised  Gibbs sampling. Applications are presented for voting records from the  United States Senate for 2003, and for the Reuters-21578 newswire collection.
Opportunities from Open Source Search.
Internet search has a strong business model that permits a free service to users, so it is difficult to see why, if at all, there should be open source offerings as well. This paper first discusses open source search, and a rationale for the computer science community at large to get involved. Because there is no shortage of core open source components for at least some of the tasks involved, the Alvis Consortium is building infrastructure for open source search engines using peer-to-peer and subject specific technology as its core, based on this rationale. We view open source search as a rich future playground in which information extraction and retrieval components can be used and intelligent agents can operate. 1
Multi%2DFaceted Information Retrieval System for Large Scale Email Archives.
We profile a system for search and analysis of largescale email archives. The system builds around four facets: Content-based search engine, statistical topic model, automatically inferred social networks and time-series analysis. The facets correspond to the types of information available in email data.
Open source search a data mining platform.
Commercial search engines provide a quality service at no cost to consumers thanks to embedded targeted marketing. Despite this, I argue there are still reasons why an open source e#ort should be encouraged in the community: as part of broader open publishing initiatives, to allow quality subject specific search engines to develop, and &#034;because we can,&#034; because search is one of the great grand challenge problems and we, the research community, cannot join in without an accessible, nonproprietary system. This talk outlines arguments and discusses some of the new technology that could go into such a system, as well as the infrastructure that would be required to make it work.
ALVIS Superpeer Semantic Search Engine.
Start date of the project 1.1.2004
Automated Synthesis of Data Analysis Programs Learning in Logic.

Applying Discrete PCA in Data Analysis.
Methods for analysis of principal components in discrete data have existed for some time under various names such as grade of membership modelling, probabilistic latent semantic analysis, and genotype inference with admixture. In this paper we explore a number of extensions to the common theory, and present some application of these methods to some common statistical tasks. We show that these methods can be interpreted as a discrete version of ICA. We develop a hierarchical version yielding components at different levels of detail, and additional techniques for Gibbs sampling. We compare the algorithms on a text prediction task using support vector machines, and to information retrieval.
A Scalable Topic%2DBased Open Source Search Engine.
Site-based or topic-specific search engines work with mixed success because of the general difficulty of the information retrieval task, and the lack of good link information to allow authorities to be identified. We are advocating an open source approach to the problem due to its scope and need for software components. We have adopted a topicbased search engine because it represents the next generation of capability. This paper outlines our scalable system for site-based or topic-specific search, and demonstrates the developing system on a small 250,000 document collection of EU and UN web pages.
Exploring Independent Trends in a Topic%2DBased Search Engine.
Topic-based search engines are an alternative to simple keyword search engines that are common in today&#039;s intranets. The temporal behaviour of the topics in a topic model based search engine can be used for trend analysis, which is an important research goal on its own. We apply topic modelling to an online financial newspaper data and show that some of the trends in the topics are consistent with common understanding.
Variational Extensions to EM and Multinomial PCA.
Several authors in recent years have proposed discrete analogues to principle component analysis intended to handle discrete or positive only data, for instance suited to analyzing sets of documents. Methods include non-negative matrix factorization, probabilistic latent semantic analysis, and latent Dirichlet allocation. This paper begins with a review of the basic theory of the variational extension to the expectation maximization algorithm, and then presents discrete component finding algorithms in that light. Experiments are conducted on both bigram word data and document bag-of-word to expose some of the subtleties of this new class of algorithms.
Automatic Derivation of Statistical Algorithms The EM Family and Beyond.
Machine learning has reached a point where many probabilistic methods  can be understood as variations, extensions and combinations of a  much smaller set of abstract themes, e.g., as different instances of the  EM algorithm. This enables the systematic derivation of algorithms customized  for different models. Here, we describe the AUTOBAYES system  which takes a high-level statistical model specification, uses powerful  symbolic techniques based on schema-based program synthesis and  computer algebra to derive an efficient specialized algorithm for learning  that model, and generates executable code implementing that algorithm.
Learning as applied to stochastic optimization for standard%2Dcellplacement.



Analysing Rock Samples for the Mars Lander.
In the near future NASA intends to explore various regions of our solar system using robotic devices such as rovers, spacecraft, airplanes, and/or balloons. Such platforms will carry imaging devices, and a variety of analytical instruments intended to evaluate the chemical and mineralogical nature of the environment(s) that they encounter. The imaging and/or spectroscopic devices will acquire tremendous volumes of data. The communication band-widths are restrictive enough so that only a small portion of these data can actually be sent to Earth.
Trends %26 Controversies Will Domain%2DSpecific Code Synthesis Become a Silver Bullet%3F

Adaptive methods for netlist partitioning.

Graphical Models for Discivering Knowledge.

A Guide to the Literature on Learning Probabilistic Networks from Data.
This literature review discusses different methods under the general rubric of learning Bayesian networks from data, and includes some overlapping work on more general probabilistic networks. Connections are drawn between the statistical, neural network, and uncertainty communities, and between the different methodological communities, such as Bayesian, description length, and classical statistics. Basic concepts for learning and Bayesian networks are introduced and methods are then reviewed. Methods are discussed for learning parameters of a probabilistic network, for learning the structure, and for learning hidden variables. The presentation avoids formal definitions and theorems, as these are plentiful in the literature, and instead illustrates key concepts with simplified examples.  Keywords--- Bayesian networks, graphical models, hidden variables, learning, learning structure, probabilistic networks, knowledge discovery. I. Introduction  Probabilistic networks or probabilistic gra...
Intelligent Instruments Discovering How to Turn Spectral Data into Information.

Chain graphs for learning.
Chain graphs combine directed and undirected graphs and their underlying mathematics combines properties of the two. This paper gives a simplified definition of chain graphs based on a hierarchical combination of Bayesian (directed) and Markov (undirected) networks. Examples of a chain graph are multivariate feed-forward networks, clustering with conditional interaction between variables, and forms of Bayes classifiers. Chain graphs are then extended using the notation of plates so that samples and data analysis problems can be represented in a graphical model as well. Implications for learning are discussed in the conclusion. 1 Introduction Probabilistic networks are a notational device that allow one to abstract forms of probabilistic reasoning without getting lost in the mathematical detail of the underlying equations. They offer a framework whereby many forms of probabilistic reasoning can be combined and performed on probabilistic models without careful hand programming. Efforts ...
On Solving Equations and Disequations.
. This paper gives a brief description of the recent improvements of lpg, a programming language that integrates algebraic and logic programming paradigms. In this language, functions are defined by means of constructor-based conditional term rewriting systems and predicates are defined by means of Horn clauses where the bodies may contain classical literals as well as equations and negations of equations (disequations). The operational semantics of lpg is based on an extension of SLD\Gammaresolution which deals with equations and disequations by using a narrowing-based algorithm. We investigate such a narrowing-based algorithm for solving equations and disequations and give new sufficient conditions that ensure the completeness of narrowing strategies, in presence of conditional term rewrite systems. Similarities and differencies with other logic and functional languages are pointed out. 1 Introduction  After the proposal of the language loglisp by Robinson and Sibert [22], many langu...
Operations for Learning with Graphical Models.
This paper is a multidisciplinary review of empirical, statistical learning from a graphical model perspective. Well-known examples of graphical models include Bayesian networks, directed graphs representing a Markov chain, and undirected networks representing a Markov field. These graphical models are extended to model data analysis and empirical learning using the notation of plates. Graphical operations for simplifying and manipulating a problem are provided including decomposition, differentiation, and the manipulation of probability models from the exponential family. Two standard algorithm schemas for learning are reviewed in a graphical framework: Gibbs sampling and the expectation maximization algorithm. Using these operations and schemas, some popular algorithms can be synthesized from their graphical specification. This includes versions of linear regression, techniques for feed-forward networks, and learning Gaussian and discrete Bayesian networks from data. The paper conclu...
Guest Editorial.
of vital signs, and medical emergency teams
A Further Comparison of Splitting Rules for Decision%2DTree Induction.
The CN2 algorithm induces an ordered list of classification rules from examples using entropy as its search heuristic. In this short paper, we describe two improvements to this algorithm. Firstly, we present the use of the Laplacian error estimate as an alternative evaluation function and secondly, we show how unordered as well as ordered rules can be generated. We experimentally demonstrate significantly improved performances resulting from these changes, thus enhancing the usefulness of CN2 as an inductive tool. Comparisons with Quinlan&#039;s C4.5 are also made. Keywords: learning, rule induction, CN2, Laplace, noise 1 Introduction Rule induction from examples has established itself as a basic component of many machine learning systems, and has been the first ML technology to deliver commercially successful applications (eg. the systems GASOIL [Slocombe et al., 1986], BMT [Hayes-Michie, 1990], and in process control [Leech, 1986]). The continuing development of inductive techniques is t...
Classifiers A Theoretical and Empirical Study.
This paper describes how a competitive tree learning algorithm can be derived from first principles. The algorithm approximates the Bayesian decision theoretic solution to the learning task. Comparative experiments with the algorithm and the several mature AI and statistical families of tree learning algorithms currently in use show the derived Bayesian algorithm is consistently as good or better, although sometimes at computational cost. Using the same strategy, we can design algorithms for many other supervised and model learning tasks given just a probabilistic representation for the kind of knowledge to be learned. As an illustration, a second learning algorithm is derived for learning Bayesian networks from data. Implications to incremental learning and the use of multiple models are also discussed. 1 Introduction Systems for learning classification trees [ Quinlan, 1986; Cestnik et al., 1987 ] are common in machine learning, statistics and pattern recognition. Despite these suc...
Some Properties of Plausible Reasoning.

Theory Refinement on Bayesian Networks.
Theory refinement is the task of updating a domain theory in the light of new cases, to be done automatically or with some expert assistance. The problem of theory refinement under uncertainty is reviewed here in the context of Bayesian statistics, a theory of belief revision. The problem is reduced to an incremental learning task as follows: the learning system is initially primed with a partial theory supplied by a domain expert, and thereafter maintains its own internal representation of alternative theories which is able to be interrogated by the domain expert and able to be incrementally refined from data. Algorithms for refinement of Bayesian networks are presented to illustrate what is meant by &#034;partial theory&#034;, &#034;alternative theory representation &#034;, etc. The algorithms are an incremental variant of batch learning algorithms from the literature so can work well in batch and incremental mode. 1 Introduction Theory refinement is the task of updating a domain theory in the light of...
Modelling default and likelihood reasoning as probabilistic reasoning.
Work reported herein was supported in part by Cooperative Agreements NCC 2-387 between the,National
Myths and Legends in Learning Classification Rules.

A Critique of the Valiant Model.

Learning Classification Rules Using Bayes.

Machine Invention of First Order Predicates by Inverting Resolution.

Generalized Subsumption and Its Applications to Induction and Redundancy.

Decision tree induction systems A Bayesian analysis.

Decision Tree Induction Systems A Bayesian Analysis.


Induction of Horn Clauses Methods and the Plausible Generalization Algorithm.


